What Makes an Item Difficult? A Syntactic, Lexical, and Morphological Study of Elicited Imitation Test Items

Elicited imitation (EI) is an oral language testing method that has been experimented with since the 1960s. Its first uses were focused on assessing first language acquisition in children (Ervin-Tripp, 2001) and diagnosing language abnormalities (Berry, 1976; Lahey, Launer, & Schiff-Myers, 1983). Later, in the 1970s, researchers began to assess how useful the method would be within the field of second language acquisition (Naiman, 1974). Since then, there have been two main focuses of Elicited Imitation studies within second language acquisition (SLA): psycholinguistic research into language competence and SLA processes and research in using the method as an indirect measurement of oral language proficiency (Bley-Vroman & Chaudron, 1994; Erlam, 2006; Vinther, 2002). We focus our research on the second, measuring oral language proficiency with the use of Elicited Imitation. Elicited imitation is a method of testing where a subject hears an utterance and then is asked to repeat the utterance they have heard. It involves both the subject’s short-term and long-term memory. The theory behind using it as measurement of oral proficiency is that a student cannot successfully repeat an utterance if he cannot understand it (Bley-Vroman & Chaudron, 1994). Essentially, a subject will hear an utterance, form a representation of that utterance using his knowledge base (long-term memory) and store that representation in his short-term memory. Then the subject will form and produce a sentence based on the representation he has created. Bley-Vroman and Chaudron state in their research of the method, ”the more you know of a foreign language, the better you can imitate the sentences of the language. Thus EI is a reasonable measure of global proficiency” (Bley-Vroman & Chaudron, 1994, p. 247). After performing several experiments with EI in an effort to determine its effectiveness in testing oral language proficiency (Graham, Lonsdale, Kennington, Johnson, & McGhee, 2008), we narrowed our focus to the design of individual test items (the sentences subjects are asked to repeat). Since second language learners must depend on their knowledge base to repeat a test item, the test items must be harder for novices than for advanced learners. The difficulty of the test item depends on the complexity of the sentence. For example, a novice learner would have difficulty repeating sentences that involve content words they are unfamiliar with, but an advanced learner who had already acquired those words would be more successful. In this study we examine the effects of different syntactic, lexical, and morphological features present in the items on their difficulty. We began by taking previously administered EI tests and determining the item difficulty based on the results using an Item Response Theory (IRT) analysis. Once this was done, each sentence was tagged for syntactic, lexical, and morphological features that could possibly affect the difficulty of an item. Once the sentences were tagged, Multiple Linear Regression analyses were done to see which factors have the most impact on item difficulty. The purpose of this study is to enlighten us as to the influences of different factors on how students perform on Elicited Imitation test items. The study provides a methodological approach to deconstructing our current test items so that we may construct new items in the future with